SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Modality (human–computer interaction)</title>

</head>

<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Modality (human–computer interaction)</span></span>

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">Not to be confused with <a href="Mode_(user_interface)" title="Mode (user interface)">Mode (user interface)</a>.</div>

<p>In the context of <a href="Human%E2%80%93computer_interaction" title="Human–computer interaction">human–computer interaction</a>, a <b>modality</b> is the classification of a single independent channel of <a href="Input/output" title="Input/output">input/output</a> between a computer and a human. Such channels may differ based on sensory nature (e.g., visual vs. auditory),<sup id="cite_ref-HCI_Overview2_1-0" class="reference"><a href="#cite_note-HCI_Overview2-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup> or other significant differences in processing (e.g., text vs. image).<sup id="cite_ref-2" class="reference"><a href="#cite_note-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup>

A system is designated unimodal if it has only one modality implemented, and <a href="Multimodal_interaction" title="Multimodal interaction">multimodal</a> if it has more than one.<sup id="cite_ref-HCI_Overview2_1-1" class="reference"><a href="#cite_note-HCI_Overview2-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup> When multiple modalities are available for some tasks or aspects of a task, the system is said to have overlapping modalities. If multiple modalities are available for a task, the system is said to have redundant modalities. Multiple modalities can be used in combination to provide complementary methods that may be redundant but convey information more effectively.<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup> Modalities can be generally defined in two forms: computer-human and human-computer modalities.

</p>

<div class="mw-heading mw-heading2"><h2 id="Computer–human_modalities">Computer–human modalities</h2></div>

<p>Computers utilize a wide range of technologies to communicate and send information to humans:

</p>

<ul><li>Common modalities

<ul><li><a href="Visual_perception" title="Visual perception">Vision</a> – computer graphics typically through a screen</li>

<li><a href="Hearing_(sense)" class="mw-redirect" title="Hearing (sense)">Audition</a> – various audio outputs</li>

<li><a href="Haptic_technology" title="Haptic technology">Tactition</a> – vibrations or other movement</li></ul></li>

<li>Uncommon modalities

<ul><li><a href="Taste" title="Taste">Gustation</a> (taste)</li>

<li><a href="Olfaction" class="mw-redirect" title="Olfaction">Olfaction</a> (smell)</li>

<li><a href="Thermoception" title="Thermoception">Thermoception</a> (heat)</li>

<li><a href="Nociception" title="Nociception">Nociception</a> (pain)</li>

<li><a href="Equilibrioception" class="mw-redirect" title="Equilibrioception">Equilibrioception</a> (balance)</li></ul></li></ul>

<p>Any human sense can be used as a computer to human modality. However, the modalities of <a href="Visual_perception" title="Visual perception">seeing</a> and <a href="Hearing_(sense)" class="mw-redirect" title="Hearing (sense)">hearing</a> are the most commonly employed since they are capable of transmitting information at a higher speed than other modalities, 250 to 300<sup id="cite_ref-Ziefle98_4-0" class="reference"><a href="#cite_note-Ziefle98-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> and 150 to 160<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup> <a href="Words_per_minute" title="Words per minute">words per minute</a>, respectively. Though not commonly implemented as computer-human modality, tactition can achieve an average of 125 wpm<sup id="cite_ref-6" class="reference"><a href="#cite_note-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup> through the use of a <a href="Refreshable_Braille_display" class="mw-redirect" title="Refreshable Braille display">refreshable Braille display</a>. Other more common forms of tactition are smartphone and game controller vibrations.

</p>

<div class="mw-heading mw-heading2"><h2 id="Human–computer_modalities">Human–computer modalities</h2></div>

<p>Computers can be equipped with various types of <a href="Input_devices" class="mw-redirect" title="Input devices">input devices</a> and sensors to allow them to receive information from humans. Common input devices are often interchangeable if they have a standardized method of communication with the computer and <a href="Affordance" title="Affordance">afford</a> practical adjustments to the user. Certain modalities can provide a richer interaction depending on the context, and having options for implementation allows for more robust systems.<sup id="cite_ref-7" class="reference"><a href="#cite_note-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup>

</p>

<ul><li>Simple modalities

<ul><li><a href="Keyboard_(computing)" class="mw-redirect" title="Keyboard (computing)">Keyboard</a></li>

<li><a href="Pointing_device" title="Pointing device">Pointing device</a></li>

<li><a href="Touchscreen" title="Touchscreen">Touchscreen</a></li></ul></li>

<li>Complex modalities

<ul><li><a href="Computer_vision" title="Computer vision">Computer vision</a></li>

<li><a href="Speech_recognition" title="Speech recognition">Speech recognition</a></li>

<li><a href="Accelerometer" title="Accelerometer">Motion</a></li>

<li><a href="Orientation_(geometry)" title="Orientation (geometry)">Orientation</a></li></ul></li></ul>

<p>With the increasing popularity of <a href="Smartphones" class="mw-redirect" title="Smartphones">smartphones</a>, the general public are becoming more comfortable with the more complex modalities. Motion and orientation are commonly used in smartphone mapping applications. Speech recognition is widely used with Virtual Assistant applications. Computer Vision is now common in camera applications that are used to scan documents and QR codes.

</p>

<div class="mw-heading mw-heading2"><h2 id="Using_multiple_modalities">Using multiple modalities</h2></div>

<div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Multimodal_interaction" title="Multimodal interaction">Multimodal interaction</a></div>

<p>Having multiple modalities in a system gives more <a href="Affordance" title="Affordance">affordance</a> to users and can contribute to a more robust system. Having more also allows for greater <a href="Accessibility" title="Accessibility">accessibility</a> for users who work more effectively with certain modalities. Multiple modalities can be used as backup when certain forms of communication are not possible. This is especially true in the case of redundant modalities in which two or more modalities are used to communicate the same information. Certain combinations of modalities can add to the expression of a computer-human or human-computer interaction because the modalities each may be more effective at expressing one form or aspect of information than others.

</p><p>There are six types of cooperation between modalities, and they help define how a combination or fusion of modalities work together to convey information more effectively.<sup id="cite_ref-:0_8-0" class="reference"><a href="#cite_note-:0-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup>

</p>

<ul><li><b>Equivalence:</b> information is presented in multiple ways and can be interpreted as the same information</li>

<li><b>Specialization:</b> when a specific kind of information is always processed through the same modality</li>

<li><b>Redundancy:</b> multiple modalities process the same information</li>

<li><b>Complementarity:</b> multiple modalities take separate information and merge it</li>

<li><b>Transfer: </b>a modality produces information that another modality consumes</li>

<li><b>Concurrency:</b> multiple modalities take in separate information that is not merged</li></ul>

<p>Complementary-redundant systems are those which have multiple sensors to form one understanding or dataset, and the more effectively the information can be combined without duplicating data, the more effectively the modalities cooperate. Having multiple modalities for communication is common, particularly in smartphones, and often their implementations work together towards the same goal, for example gyroscopes and accelerometers working together to track movement.<sup id="cite_ref-:0_8-1" class="reference"><a href="#cite_note-:0-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup>

</p>

<ul><li><a href="Multimodal_learning" title="Multimodal learning">Multimodal learning</a> – Machine learning methods using multiple input modalities</li>

<li><a href="Multisensory_integration" title="Multisensory integration">Multisensory integration</a> – Study of senses and nervous system</li>

<li><a href="User_interface" title="User interface">User interface</a> – Means by which a user interacts with and controls a machine</li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFKarrayAlemzadehSalehArab2008" class="citation journal cs1">Karray, Fakhreddine; Alemzadeh, Milad; Saleh, Jamil Abou; Arab, Mo Nours (March 2008). <a rel="nofollow" class="external text" href="https://web.archive.org/web/20150430205510/http://s2is.org/Issues/v1/n1/papers/paper9.pdf">"Human-Computer Interaction: Overview on State of the Art"</a> <span class="cs1-format">(PDF)</span>. <i>International Journal on Smart Sensing and Intelligent Systems</i>. <b>1</b> (1): <span class="nowrap">137–</span>159. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.21307%2Fijssis-2017-283">10.21307/ijssis-2017-283</a>. Archived from <a rel="nofollow" class="external text" href="http://www.s2is.org/issues/v1/n1/papers/paper9.pdf">the original</a> <span class="cs1-format">(PDF)</span> on April 30, 2015<span class="reference-accessdate">. Retrieved <span class="nowrap">April 21,</span> 2015</span>.</cite></span>

</li>

<li id="cite_note-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-2">^</a></b></span> <span class="reference-text"><cite id="CITEREFJing_Yu_KohSalakhutdinovFried2023" class="citation arxiv cs1">Jing Yu Koh; Salakhutdinov, Ruslan; Fried, Daniel (2023). "Grounding Language Models to Images for Multimodal Inputs and Outputs". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/2301.13823">2301.13823</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>

</li>

<li id="cite_note-3"><span class="mw-cite-backlink"><b><a href="#cite_ref-3">^</a></b></span> <span class="reference-text"><cite id="CITEREFPalanquePaterno2001" class="citation book cs1">Palanque, Philippe; Paterno, Fabio (2001). <a rel="nofollow" class="external text" href="https://archive.org/details/springer_10.1007-3-540-44675-3"><i>Interactive Systems. Design, Specification, and Verification</i></a>. Springer Science & Business Media. pp. <a rel="nofollow" class="external text" href="https://archive.org/details/springer_10.1007-3-540-44675-3/page/n50">43</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9783540416630</bdi>.</cite></span>

</li>

<li id="cite_note-Ziefle98-4"><span class="mw-cite-backlink"><b><a href="#cite_ref-Ziefle98_4-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFZiefle1998" class="citation journal cs1">Ziefle, M (December 1998). "Effects of display resolution on visual performance". <i>Human Factors</i>. <b>40</b> (4): <span class="nowrap">554–</span>68. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1518%2F001872098779649355">10.1518/001872098779649355</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/9974229">9974229</a>.</cite></span>

</li>

<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text">Williams, J. R. (1998). Guidelines for the use of multimedia in instruction, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1447–1451</span>

</li>

<li id="cite_note-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-6">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.acb.org/node/67">"Braille"</a>. <i>ACB</i>. American Council of the Blind<span class="reference-accessdate">. Retrieved <span class="nowrap">21 April</span> 2015</span>.</cite></span>

</li>

<li id="cite_note-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-7">^</a></b></span> <span class="reference-text"><cite id="CITEREFBainbridge2004" class="citation book cs1">Bainbridge, William (2004). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=568u_k1R4lUC"><i>Berkshire Encyclopedia of Human-computer Interaction</i></a>. Berkshire Publishing Group LLC. p. 483. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9780974309125</bdi>.</cite></span>

</li>

<li id="cite_note-:0-8"><span class="mw-cite-backlink">^ <a href="#cite_ref-:0_8-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-:0_8-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFGrifoni2009" class="citation book cs1">Grifoni, Patrizia (2009). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=O8CqMtIKSWwC"><i>Multimodal Human Computer Interaction and Pervasive Services</i></a>. IGI Global. p. 37. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781605663876</bdi>.</cite></span>

</li>

</ol></div></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-03-29" href="https://en.wikipedia.org/wiki/?title=Modality_(human%E2%80%93computer_interaction)&oldid=1282991989">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>